Explicit Approximations of the Gaussian Kernel

We investigate training and using Gaussian kernel SVMs by approximating the kernel with an explicit finitedimensional polynomial feature representation based on the Taylor expansion of the exponential. Although not as efficient as the recently-proposed random Fourier features [Rahimi and Recht, 2007] in terms of the number of features, we show how this polynomial representation can provide a better approximation in terms of the computational cost involved. This makes our “Taylor features” especially attractive for use on very large data sets, in conjunction with online or stochastic training.